Week 9: Dimensionality Reduction

In the preceding lectures, we discussed clustering. One view of clustering is that it’s a way to summarize a complex real-valued datapoint x ∈ R with a single categorical variable y ∈ {1, . . . ,K}. This could be useful for understanding and visualizing the structure in the data, as well as a preprocessing step for other learning algorithms. For example, we could build a very simple classifier on top of y instead of dealing with all the complexity of x. In this lecture, we’ll discuss another way to simplify complex, high-dimensional data for visualization or subsequent (supervised learning), where instead of summarizing the datapoint x with a categorical variable y, we instead summarize it with a smaller realvalued vector z. For example, x might represent all of the pixels in an image of a face, or all of the vertices in a 3D scan of a person’s body – thousands or millions of dimensions – while z might consist of only a small number of parameters, such as the person’s height and weight, or the direction the face in the image is pointing. Summarizing continuous high-dimensional datapoints with continuous low-dimensional datapoints is called dimensionality reduction. The lecture slides show a few examples of dimensionality reduction problems.